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dear 


Our  18  year  histoiy  in  HIV  has  yielded  many  lessons  learned  from  the  field.  Funding  for 
HIV  programs  have  not  increased  proportionally  with  the  level  of  need  for  communities  of 
color.  We  have  acknowledged  that  the  HIV  virus  knows  no  color  and  that  we  need  to  create 
and  maintain  programs  that  "work."  Programs  that  "work"  need  evidence  of  why  they  work. 
Now  more  than  ever,  we  need  to  integrate  Evaluation  with  our  HIV  Prevention  programs.  We 
need  to  evaluate  why  our  programs  work  and  the  nuances  of  our  cultures  that  are  enablers  or 
barriers  to  making  our  programs  work. 

The  race  for  funding  has  become  an  endurance  marathon,  and  this  is  the  race  we  can  lead 
because  we  understand  and  respect  the  nuances  of  our  own  cultures. 

Since  1987,  the  National  Minority  AIDS  Council  (NMAC)  has  been  committed  to 
developing  leadership  within  communities  of  color  to  address  the  challenges  of  HIV  infection. 
Our  work  in  technical  assistance  and  training  is  a part  of  this  commitment.  As  part  of  this  work 
we  acknowledge  the  need  for  evaluation  as  the  precursor  for  planning  your  programs. 

We  would  like  to  thank  Charlene  Day  for  the  creation  of  this  document,  and  deep 
appreciation  to  Virginia  Bourassa  for  her  involvement  in  the  development  process.  Kudos  also 
to  Faith  Hunter  Kephart  for  oversight  of  the  program  activity  and  assurance  that  the  goal  was 
obtainable.  A special  thanks  to  The  Gann  Agency  for  their  design  insights  and  expertise  and 
many  thanks  to  the  Centers  for  Disease  Control  for  their  funding  support.  v/ 

This  document  is  dedicated  to  the  staff  and  volunteers  on  the  front  lines  and  in  the 
communities  who  are  on  the  pulse  of  the  epidemic. 

Yours  in  the  struggle, 


Paul  Kawata 
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introduction 


Throughout  the  past  five  years,  NMAC  has  been  a 
recipient  of  a grant  from  the  Centers  For  Disease  Control  and 
Prevention  National/Regional  Minority  Organizations 
Cooperative  Agreement  Program  to  provide  technical 
assistance  (TA)  and  training  to  community-based 
organizations  that  serve  racial,  ethnic  and  undeserved 
minority  populations  at  risk  for  HIV  and  other  STDs.  Early 
in  our  project,  we  wanted  to  know  if  our  TA  efforts  were 
useful,  and  whether  or  not  our  provision  of  advice  and 
expertise  made  a difference  to  the  many  grantees  to  whom  we 
provided  services.  Equally  important,  we  wanted  to  "practice 
what  we  preach"...  mainly  that  you  must  commit  time,  money 
and  effort  to  evaluation  at  the  very  beginning  of  a project.  In 
order  to  answer  these  questions,  we  employed  an  external 
evaluator. 

Using  an  evaluator  has  been  an  important  step  in  the 
development  of  the  TA  Division.  We  have  been  able  to 
capture  data  and  to  list  success  stories,  one  after  the  other, 
because  of  our  evaluation  plan.  As  a result  of  our  evaluation 
efforts  we  can  tell  you  the  number  and  type  of  TA  delivered 
over  the  course  of  our  grant  period  and  the  perceived 
usefulness  of  that  assistance.  By  the  years  end,  we  will  be 
able  to  tell  you  whether  the  provision  of  TA  made  a difference 
in  the  problem  area  cited  by  the  agency.  For  example,  if  the 
agency  sought  TA  in  board  development,  our  evaluation  will 
help  us  to  know  if  a board  of  directors  has  been  established 
within  one  year  after  the  provision  of  our  TA. 

Evaluation  can  also  be  an  instructive  tool  which  can  help 
Managers  adjust  or  enhance  its  program  throughout  the 
intervention.  For  example,  through  evaluation,  NMACs  TA 
division  was  able  to  demonstrate  the  regional  impact  of  their 
technical  assistance  program  across  the  United  States.  In  the 
third  year  of  the  provision  of  grants,  NMAC  provided 
technical  assistance  to  30  agencies.  Technical  assistance  was 
provided  to  two  organizations  in  Hawaii,  three  in  the 
midwestem  states  eight  in  the  northeastern  states,  ten  in  the 
southern  states,  five  in  the  western  states  and  two  in  Puerto 
Rico.  These  agencies  represented  six  regions  throughout  the 
United  States,  including  the  U.S.  territory,  Puerto  Rico. 
Having  this  information  was  useful  to  the  NMAC  staff  as 
they  further  developed  strategy  to  expand  their  technical 
assistance  to  populations  in  need.  NMAC  used  this 

information  to  enhance 
advertising,  and  expand  the  range 
of  request  for  proposals 
disseminated.  These  efforts 
resulted  in  an  expanded  service 


population  by  the  fifth  year  of  its  grant. 

At  the  conclusion  of  its  five  year  cooperative  agreement 
with  the  federal  Centers  For  Disease  Control  and  Prevention, 
(1998),  NMAC  has  provided  technical  assistance  grants  to 
over  120  HIV/AIDS  prevention  organizations  in  over  31 
states.  NMACs  evaluation  has  also  enabled  the  staff  to  see 
changes  in  the  needs  of  technical  assistance  grantees  over  the 
five  year  period.  In  the  third  year  of  NMAC's  technical 
assistance  grants,  a great  demand  was  made  for  assistance  in 
the  area  of  board  development.  However,  in  the  04  year  of 
the  provision  of  grants  for  technical  assistance,  NMAC 
witnessed  a shift  in  priorities  stated  by  grantees  to  resource 
development.  This  area  remains  the  most  frequently 
requested  area  for  assistance  in  the  05  year  of  technical 
assistance  grants. 

I hope  that  through  this  demonstration  of  the  impact  of 
evaluation,  that  you  may  begin  to  see  its  necessity  and 
usefulness  to  the  program  planner.  Armed  with  only  this 
rudimentary  form  of  evaluation,  NMAC  could  be  poised  to 
request  additional  funds  from  its  grantor  because  it  can  prove 
there  is  a need  for  technical  assistance  in  specific  areas. 
NMAC  staff  could  also  tailor  their  technical  assistance 
programs  by  employing  consultants  in  the  area  in  which 
needs  are  being  expressed,  and  continue  to  respond  in  a 
highly  organized,  user  friendly  manner  to  technical  assistance 
grantees. 

Evaluation  findings  may  not  always  be  positive,  or 
statistically  significant,  but  the  data  is  almost  always  useful. 
We  urge  you  to  read  this  manual  and  to  refer  to  it  frequently 
as  you  contemplate  the  evaluation  needs  of  your  HIV/AIDS 
prevention  program.  The  manual  is  divided  into  the  three 
sections  described  below: 

Stages  and  Phases  Of  Evaluation 

This  article,  written  by  our  Project  Evaluator,  Dr.  Day 
provides  an  overview  of  the  stages  and  phases  of  evaluation 
that  many  HIV/AIDS  prevention  organizations  encounter. 
While  examples  are  given  about  the  size,  type  of  agency, 
services  rendered  and  length  of  time  in  the  HIV/AIDS 
prevention  arena,  this  information  should  only  be  viewed  as  a 
guide.  Your  organization  may  or  may  not  have  some  of  these 
qualities.  In  order  for  this  section  to  be  instructive  and 
useful,  the  reader  should  not  become  mired  in  the 
organizational  classification,  but  rather  focus  on  the 
evaluation  dynamics  for  each  phase,  and  the  action  plan  for 
evaluation  improvement. 

continued  on  next  page 


Evaluation  Instruments 

This  compilation  of  actual  evaluation  instruments  used 
by  HIV/AIDS  prevention  organizations  provides  the  reader 
with  a potpourri  of  tools  that  can  be  used  in  evaluating 
HIV/AIDS  programs.  These  instruments  are  provided  so 
that  organizations  may  review  and  adapt  data  collection 
tools  that  are  being  used  by  other  prevention  programs. 

Glossary  of  Terms 

A glossary  of  terms  is  provided  to  assist  the  reader  in 
understanding  the  plethora  of  terms  used  throughout  this 
document. 

NMAC  Survey 

Mentioned  previously  in  this  introduction,  NMAC’s  TA 


staff  seeks  to  practice  what  we  preach.  Thus  we  have 
included  a brief  survey  seeking  your  input  on  the  usefulness 
of  this  manual  and  your  perceptions  about  evaluation  efforts 
in  your  agency.  Please  take  a few  minutes  to  assist  us  in  our 
evaluation  efforts  by  completing  this  survey  and  mailing  it  to 
us  today.  All  that  is  required  is  your  anonymous  response 
and  placement  of  the  completed  survey  in  your  local 
mailbox. 

Note:  NMAC’s  evaluation  includes  both  process  and 
outcome  measures.  If  you  are  interested  in  other  measures 
used  by  NMAC’s  Technical  Assistance  Division,  please  feel 
free  to  contact  Dr.  Day  at  (202)  291-8975. 


the  three  phases  of  evaluation 


Throughout  the  past  decade,  government 
agencies,  along  with  national  and  community- 
based  organizations  have  worked  diligently  to 
prevent  increases  in  the  number  of  people 
infected  with  HIV/AIDS.  As  resources  — 
financial  and  human  — become  increasingly 
competitive,  the  HIV/AIDS  prevention 
community  has  become  acutely  aware  of  the 
need  to  demonstrate  the  value  of  its 
prevention  efforts.  By  engaging  in  rigorous 
evaluation  of  prevention  efforts,  organizations 
can  assess  whether  change  in  attitudes,  beliefs 
and  behaviors  has  occurred,  and  whether  that 
change  can  be  attributed  to  a specific 
prevention  effort.  The  description  of  the 
various  phases  of  evaluation  in  HIV/ AIDS 
prevention  efforts  are  described  below,  along 
with  suggested  action  steps  organizations  can 
take  to  enhance  evaluation  strategies. 

PHASES  OF  HIV/AIDS  PROGRAM  EVALUATION 

^ Phase Phase Phase 

1 2 3 

Start-up  Multiple  Multi-level 

programs  (Multiple  programs 

and  priorities) 


Phase  1,  Phase  2 and  Phase  3 are  points  at 
which  agencies  and  organizations  can  assess 
their  progress  towards  developing  and 
implementing  evaluation  plans  for  HIV/AIDS 
prevention  programs.  The  phases  described 
are  fluid  and  dynamic;  therefore  organizations 
may  find  elements  of  themselves  in  one  or 
more  of  these  three  phases.  The  evaluation 
descriptions  provided  seek  to  capture  the 
essence  of  an  HIV/AIDS  prevention  effort  at 
a specific  time.  While  reviewing  each  of  the 
three  phases,  HIV/AIDS  Program  Managers 
and  staff  may  see  aspects  of  their  organization 
in  one  or  more  phase.  Try  to  focus  on  the 
phase  that  most  accurately  represents  where 
your  organization  is  in  its  development  of  an 
evaluation  strategy. 

PLEASE  NOTE:  While  review ng  each  of  the  three 
phases,  HIV/ AIDS  Program  Managers  and  staff  may 
recognize  similarities  in  some  or  all  aspects  of  a 
particular  phase.  Do  not  limit/restrict  yourself  to  one 
phase  simply  because  the  initial  demographic  do  not 
fit  your  organization's  current  status. 


phase  one  evaluation  in  HIV/AIDS  prevention  programs 


Description 

Organizations  in  the  "Phase  One"  stage 
of  evaluation  tend  to  be  described  by  their 
passion  and  commitment  to  HIV/AIDS 
prevention.  In  many  cases,  an  organization  in 
the  "Phase  One"  stage  of  evaluation  may  be  a 
new  or  start-up  community  based 
organization  (CBO).  They  may  also  be 
organizations  that  evolved  from  a "grassroots" 
experience  and  have  become  so  involved  in 
the  provision  of  services  that  lack  the  staff  or 
resources  to  shift  focus  to  evaluation.  This 
type  of  CBO  may  have  been  created  for  the 
sole  purpose  of  decreasing  the  rate  of  newly 
infected  persons  with  HIV  in  the  community. 
Very  often,  the  funding  they  have  received  is 
stipulated  for  use  in  outreach  and  education 
efforts  only.  A majority  of  the  staff  may  be 
volunteers.  In  many  cases,  CBOs  have 
received  funding  because  they’ve  made  a 
persuasive  argument  that  their  particular 
services  are  worthwhile  and  will  produce  a 
desired  outcome.  However,  "proving"  this 
relationship  between  the  program  and 
outcome  is  generally  a theoretical  argument. 

Generally  staff  organizations  in  the 
"Phase  One"  stage  of  evaluation  have  as  their 
primary  focus  the  dissemination  of 
information,  education  materials,  and  the 
distribution  of  condoms.  Organizations  in  the 
"Phase  One"  stage  may  also  be  conducting 
HIV/AIDS  101  and  201  workshops 
throughout  the  community,  making  referrals 
to  service  related  agencies  and  working 
collaboratively  with  organizations  within  their 
communities. 

LIMITATIONS 


flyers,  condoms  and  other  information  in  the 
field.  Organizations  in  the  "Phase  One"  stage 
of  evaluation  organizations  may  or  may  not 
have  a requirement  in  grant  and  funding 
resources  to  conduct  evaluation.  The  expense 
— financially  and  otherwise  — may  appear  to 
be  unwarranted.  It  is  not  unusual  for 
organizations  in  "Phase  One"  evaluation  to 
indicate  we  simply  want  to  do  the  job  that  we 
are  receiving  money  to  do.  Weighting  us  down 
with  counting  the  number  of  condoms  and 
brochures  we  disseminate  takes  away  staff 
from  the  valuable  role  they  play  out  in  the 
field.  More  often  than  not,  however, 
organizations  in  the  "Phase  One"  stage  of 
evaluation  may  contract  with  an  evaluator  to 
develop  the  grant  proposal,  or  the  evaluation 
component  of  the  grant  proposal  only. 

This  often  the  only  point  of  contact  the 
organization  has  with  an  evaluator. 

ENHANCING  YOUR  EVALUATION  STRATEGY 

In  order  to  continue  to  work  and  receive 
funds  in  the  highly  competitive  HIV/AIDS 
prevention  arena,  CBO’s  are  increasingly 
being  called  upon  to  document  or  evaluate 
prevention  efforts.  This  mandate  requires  that 
even  the  most  rudimentary  CBO  begin  to 
systematically  collect  data  to  prove  its 
existence,  validate  that  prevention  efforts  have 
occurred,  and  begin  to  demonstrate  whether 
or  not  the  prevention  efforts  have  made  an 
impact  on  the  target  population  serviced. 

The  information  provided  below  can  assist 
HIV/AIDS  prevention  organizations  in 
“Phase  One”  stage  of  evaluation  to  enhance 
their  evaluation  efforts. 


Because  groups  of  people,  organizations 
or  agencies  in  "Phase  One"  evaluation  may  be 
new  or  in  a start-up  category,  staff  is  usually 
more  concerned  with  conducting  outreach 
efforts,  which  may  include  dissemination  of 


evaluation  action  steps  (process) 


Meet  with  ail  staff,  inducting  volunteers,  to  discuss  the  value  of  evaluation  (See  NMAC  TA  Newsletter  July 
1996).  Engage  in  an  exercise  that  reguires  each  staff  person  to  brainstorm  and  describe  what  they  believe  to 
be  the  major  goal,  objectives  and  activities  of  your  HIV/AIDS  prevention  program.  This  process  is  sometimes 
referred  to  as  defining  GOAM,  an  acronym  for  Goals,  Objectives,  Activities,  and  Milestones. 

Goals 


Objectives 

Activity 

Milestones 


Develop  an  evaluation  plan.  Together  with  staff,  develop  a plan  which  outlines  the  aforementioned 
planning  and  evaluation  strategy  (GOAM)  using  the  5 Ws  to  describe  the  full  program. 


Describe  when  you  conduct  activities  related  to  your  objective  and  under  what  conditions. 


Who  benefits  from  the  prevention  services?  Describe  the  population  in  terms  of  demographics, 
need,  and  response  levels;  compare  them  to  pre-existing  prevention  efforts. 


What  specifically  is  your  prevention  program  doing?  Are  you  trying  to  increase  knowledge  regarding 
behaviors  that  put  individuals  at  risk  for  HIV/AIDS?  Are  you  trying  to  provide  condoms  and 
hopefully  teaching  how  to  use  condoms  properly  to  a select  population?  Outline  specifically  what  it 
is  you  are  doing  and  the  rationale  for  each  using  this  approach  over  another. 


Who 


What 


When 


A statement  that  indicates  what  a program  is  supposed  to  produce.  A goal  statement  describes  the 
intended  consequences  (sometimes  referred  to  as  outcomes)  of  the  program  being  developed. 


Tasks  that  must  be  completed  to  achieve  the  goal. 


The  specific  tasks/function  undertaken  to  accomplish  the  objective 


A significant  point  in  the  development  towards  the  goal. 


Where 


Why 


Describe  the  places  in  which  your  HIV/AIDS  prevention  activities  might  occur. 
For  example:  Do  you  conduct  HIV/AIDS  101  workshops  in  beauty  and  barber 
shops  between  the  timeframes  of  5-9  on  Friday  evenings. 


This  question  sets  the  evaluation  in  motion.  What  are  the  particular  questions  the 
funders  want  you  to  answer  about  the  population  in  need?  If  you  are  only  interested 
in  whether  or  not  knowledge  was  changed  as  a direct  result  of  your  AIDS  101,  then  a 
pre/post  workshop  survey  might  suffice.  However,  if  you  are  interested  in  knowing 
whether  or  not  the  individual  plans  to  change  behavior  as  a result  of  your  workshop, 
a different  type  of  survey  and  intervention  methodology  might  be  employed. 
Additionally,  organizations  must  question  the  theory  or  approach  selected.  Why  did 
the  organization  select  one  approach  over  another?  Is  there  any  "proof’  in  the 
professional  literature  (joumals/articles)  which  supports  your  use  of  this  approach. 


Conduct  brief  surveys  before  and  after  the  prevention  workshop  (pre/post  tests).  DO  NOT  REINVENT  THE 
WHEEL  Federal,  state  and  local  health  departments,  the  National  Prevention  Information  Network  (NPIN), 
and  fellow  CBO's  may  have  already  developed  and  validated  a pre/post  test  to  be  used  in  basic  HIV/ AIDS 
workshops.  Test  the  appropriateness  of  the  instrument  for  your  selection  population  by  administering  the  test 
and  asking  for  feedback  regarding  ease  of  administration,  and  level  of  understanding.  This  approach  to  survey 
validation  is  called  a pilot. 

Collect  and  tally  demographic  information  from  individuals  who  receive  brochures  information  packets  and 
condoms. 

Develop  quarterly  reports  for  funders,  board  members  and  staff  detailing  the  organization's  progress 
towards  reaching  stated  goals  and  objectives.  Within  the  reports,  provide : 1.  a description  of  the 
approaches  used  to  reach  each  objective,  2.  an  outline  of  the  major  prevention  activities  undertaken  within  the 
quarter,  3.  a listing  of  challenges  and/or  limitations  throughout  the  quarter,  and;  4.  a discussion  of  strategies 
to  redress  the  limitations. 


Record  minutes  of  meetings  with  staff,  board  members  and  community  members. 


phase  two  evaluation  in  HIV/AIDS  prevention  programs 


description 

Organizations  in  the  "Phase  Two"  stage  of 
evaluation  are  best  described  by  their  older 
sibling  status.  They  are  no  longer  the  new  kids 
on  the  block  and  may  have  more  than  one 
issue  area  for  which  they  receive  significant 
external  funding.  Organizations  and  agencies 
in  Phase  Two  stage  evaluation  may  be 
conducting  outreach,  referral  and  treatment 
services  in  addition  to  their  prevention 
services.  Staff  includes  full  time,  professionally- 
trained  health  educators,  salaried  and  hourly 
wage  workers,  and  volunteers.  The  passion  for 
HIV/AIDS  prevention  exists,  but  is  tempered 
with  the  realities  of  participation  in  the 
HIV/AIDS  arena  for  many  years.  This  reality 
bespeaks  problems  that  are  constant.  There  is 
the  challenge  of  retaining  staff  who  often  lose 
energy  and  enthusiasm  working  in  an  arena 
which  is  stress  ridden,  labor  intensive,  and 
emotionally  draining.  Additionally,  there  is  the 
fluctuations  in  funding  cycles  which  sometimes 
reflect  the  greater  priority  to  fund  one  risk 
category  or  selected  population  over  another. 

Oftentimes,  organizations  in  “Phase  Two” 
have  been  conducting  process  and  outcome 
based  evaluation  over  the  course  of  a three  to 
five  year  grant.  These  organizations  or  agencies 
may  use  epidemiological  data  collected  by  their 
staff  or  at  the  local  and  state  health  department 
level  to  justify  the  need  for  prevention  services 
they  render.  They  may  use  state  developed 
instruments  including  self  report  behavioral 
measurement  surveys,  pre/post  workshop 
surveys  and  focus  groups  to  ascertain  changes 
in  the  population  for  whom  prevention  services 


are  targeted.  Organizations  in  Phase  Two  may 
have  allocated  money  within  grants  and 
through  general  staff  budgets  to  hire  an  internal 
evaluator. 

LIMITATIONS 

Organizations  and  agencies  in  "Phase 
Two"  evaluation  may  become  victims  of  the 
"I  know  what  works"  mindset.  Because  they 
have  worked  with  a select  population  for  a few 
years,  they  may  develop  a dominant  attitude, 
and  this  can  impact  program  evaluation. 

The  organization  may  also  begin  to  perceive 
that  since  they  have  worked  with  a particular 
population  for  a period  of  time,  then  their 
knowledge  and  opinions  give  them  the 
inherent  right  to  create  prevention  programs. 

If  this  rigid  type  of  thinking  continues,  it  might 
result  in  a stagnancy  of  prevention  programs  in 
an  evolving  disease. 

Enhancing  your  evaluation  strategy 

The  aforementioned  thinking  can  prove 
problematic  to  evaluation  planning  and 
implementation  if  it  does  not  prompt  Program 
Managers  to  consider  some  of  the  following 
questions: 

How  do  you  know  your  intervention  strategy  and 
related  activities  are  the  most  effective  one  for  the 
population  in  need? 

Have  you  used  health  education  or  health  behavior 
theory  to  develop  to  undergird  your  evaluation  plan? 


continued  on  next  page 
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Have  you  taken  steps  to  insure  that  the  instruments 
you  are  using  are  culturally  appropriate  and  sensitive 
to  gauge  the  knowledge , beliefs  and  behaviors  of  the 
population  in  need? 

Have  the  instruments  you  elected  to  use  from  the 
state  and/or  other  CBOs  been  tested  and  proven  to 
be  sound  on  your  selected  population  in  need? 

Have  you  considered  that  using  an  internal  evaluator 
only  may  lead  to  potential  bias? 

Have  you  utilized  a variety  of  evaluation  strategies, 
quantitative  and  qualitative,  to  gather  information 
and  validate  findings? 

How  do  your  posters,  flyers,  and  other  outreach 
efforts  are  being  perceived  and  received  by  the 
population  in  need? 

Have  you  conducted  follow-up  evaluation  regarding 
receptivity  of  your  instruments? 

Have  you  assessed  whether  or  not  your  organization 
is  engaging  in  both  impact  and  outcome  based 
evaluation? 

How  reflective  is  your  staff  (from  the  Executive 
Director  to  the  volunteers)  of  the  population  in  need, 
and  does  this  make  a difference  in  the  receptiveness 
and  honesty  in  reporting  of  information  from  your 
selected  population? 

Evaluation  action  Steps 

Consider  whether  or  not  your 
organization/agency  can  attribute  a 
change  in  knowledge,  beliefs  or  behavior  to 


your  specific  intervention.  How  do  you  know 
it  was  your  program,  and  not  something 
external  which  made  the  difference? 

The  discipline  of  health  education  provides 
excellent  training  on  how  to  develop, 
implement  and  evaluate  health  enhancement 
and  disease  prevention  initiatives. 

As  organizations  and  agencies  move 

deeper  into  HIV  prevention,  they  are 
finding  that  more  proof  is  required  to 
demonstrate  that  the  strategy  was  appropriate 
for  the  population  in  need.  Increasingly, 
funders  are  seeking  to  know  the  theory’s  that 
the  organization  used  to  develop  programs. 

If,  however,  the  agency  has  developed  a new 
theory,  it  is  expected  that  theory  will  be 
grounded  in  sound  research  and  tested  for 
both  reliability  and  validity.  Once  again,  the 
discipline  has  developed  and  validated 
theories  on  health  behavior  change  as  well  as 
borrowed  from  other  health  related 
disciplines  to  compile  a broad  list  of  theories 
which  may  explain  under  what  circumstances 
an  individual  may  change  his/her  behavior. 
Review  existing  theories  on  health  behavior 
and  consider  validating  it  on  the 
organization’s  population  in  need.  If  the 
theory  does  not  work,  contrary  to  popular 
opinion,  this  does  not  constitute  a negative 
finding.  In  fact,  finding  out  that  a theory  may 
not  work  on  a specific  population  in  need  is 
valuable  to  the  research  field. 

While  many  CBOs  have  developed 

instruments  to  assess  needs  and  strengths 


always  be  appropriate  for  your  population  in 
need,  who  is  to  say  an  instrument  that  works 
well  and  gathers  accurate  data  for  one 
population  will  do  the  same  for  another.  For 
example,  an  instrument  that  gathers  sexual 
behavior  among  gay  black  men  in  San 
Francisco,  California  may  not  be  effective  on 
gay  black  men  in  Birmingham,  Alabama. 
Sometimes,  minor  changes  in  terminology, 
phraseology  and  syntax  may  be  required  to 
make  it  appropriate  for  your  selected 
population.  Always  pilot  an  instrument  with  a 
subset  of  your  population  in  need  before 
implementing  the  survey  on  a large  scale. 

Using  varied  evaluation  techniques  helps 

to  insure  that  you  have  truly  understood 
the  measure  your  population  in  need  is  trying 
to  convey  about  your  intervention.  For 
example  if  you  combine  qualitative  research, 


say  a focus  group  for  example,  before 
developing  a needs/strength  assessment,  you 
will  have  a good  sense  of  what  the  community 
will  respond  to  on  a written  survey.  Indeed, 
you  may  also  gain  knowledge  about  whether  or 
not  they  will  inclined  to  respond  to  a written 
survey.  Once  the  survey  has  been  administered, 
and  the  quantitative  data  analysis  conducted,  it 
would  be  useful  to  follow-up  the  survey  with 
another  qualitative  approach  of  key  informant 
interviews  which  could  be  used  to  probe 
deeper  into  why  a specific  response  may  have 
been  given  to  a specific  question. 

Consider  hiring  someone  on  staff  to 

evaluate  the  effectiveness  of  your  program. 
Allocate  a portion  of  your  grant  to  instrument 
research,  development,  and  validation. 


m 


phase  three  evaluation  in  HIV/AIDS  prevention  programs 


Description 

Organizations  and  agencies  in  the  "Phase 
Three"  stage  of  evaluation  may  house 
HIV/AIDS  prevention  and  treatment 
programs  as  well  as  initiatives  on  Alcohol  and 
Substance  Abuse,  Violence  and  Housing  for 
the  Homeless.  These  organizations  may  focus 
on  multiple  health  and  social  service  related 
concerns  and  may  have  up  to  25  projects 
being  conducted  simultaneously,  targeting  the 
same  community  in  need,  and  receiving 
various  funding  from  numerous  foundations, 
federal  and  state  agencies  and  local 
organizations. 

Organizations  in  the  "Phase  Three"  stage 
of  evaluation  are  frequently  large  organization 
with  multiple  programs  and  multiple 
priorities.  These  organizations  have  also  been 
doing  HIV/ AIDS  prevention  for  some  time. 
Organizations  in  this  phase  may  have  a 
different  set  of  descriptors  than  others 
referenced  above.  They  may  be:  a)  national 
and  regional  organizations  receiving  funding 
by  the  Centers  for  Disease  Control  and 
Prevention  (CDC)  or  other  funders,  b) 
community  -based  organizations  who  have 
been  involved  in  HIV/AIDS  prevention, 
service  provision  and  treatment  for  years, 
and/or  c) not-for-profit  and  for-profit 
organization  who  have  partnered  with 
community  entities  to  incorporate  HIV/AIDS 
into  a larger  prevention  effort. 

LIMITATIONS 

Organizations  and  agencies  in  the  “Phase 
Three”  stage  evaluation  are  sometimes 


plagued  by  funding  constraints,  conflicting 
priorities,  and  inconsistent  messages  from  the 
community  in  need,  and  an  administrative 
agenda  which  may  not  fully  embrace  the  need 
for  rigorous  evaluation.  This  attitude  among 
administrators  may  exist  even  when  funders 
have  indicated  that  organizations  must 
allocate  a specified  amount  to  evaluation  of 
individual  prevention  programs. 

Enhancing  your  evaluation  strategy 

Program  planners  may  have  to  consider 
some  of  the  following  questions  when 
attempting  to  enhance  existing  evaluation 
strategy. 

Organizations  in  "Phase  Three"  evaluation  must 
collect  outcome  data.  What  were  the  results  of  your 
intervention  ? Did  you  change  attitudes  or  behavior 
with  your  specific  intervention? 

Organizations  in  "Phase  Three"  frequently  participate 
in  cooperative  agreements  with  funding  entities 
which  require  them  to  work  in  concert  and 
cooperation  with  the  funder.  Are  there  significant 
differences  between  the  desired  outcomes  of  the 
funder  and  of  the  community  in  need? 

Organizations  in  " Phase  Three"  may  be  further 
removed  from  the  immediate  population  in  need  and 
therefore  may  face  unique  challenges  in  data 
collection.  Is  your  proximity,  physically  and 
philosophically  consistent  with  the  community  in 
need? 

Evaluation  Action  Steps:  4A's 

Allocate  funds:  Consider  allocating 

funds,  (10-15%  of  the  grant)  for  an 
external  evaluator.  Try  to  make  certain  that 


you  select  someone  who  has  had  specific 
background  and  experience  in  HIV/AIDS 
program  evaluation,  or  at  the  very  least  is  a 
Certified  Health  Education  Specialist  with  an 
emphasis  in  evaluation. 

Acknowledge  bias:  While  using  an 

internal  evaluator  is  an  important  step  in 
the  right  direction  towards  evaluation  of  your 
HIV/AIDS  prevention  program,  there  exists  a 
certain  bias  that  is  inherent  in  internal 
evaluation.  Very  often  internal  evaluators 
have  more  than  one  program  to  which  they 
are  assigned.  Staff  are  frequently  competing 
for  the  attention  of  the  evaluator,  and 
sometimes  set  guidelines  which  are  difficult 
with  which  to  comply.  Equally  important  is 
the  fact  that  the  internal  evaluator  is  a part  of 
the  staff  and  therefore  wants  the 
program/project  to  succeed.  This  bias  might 
be  reflected  in  the  reporting  and  interpretation 
of  findings.  For  example,  an  internal  evaluator 
conducting  a focus  group  which  is  seeking  to 
ascertain  perceptions  of  the  community  about 
your  AIDS  101  classes  may  find 
himself/herself  in  a compromising  position. 

He  or  she  wants  to  gamer  the  information  but 
may  bias  the  group  responses  because  of 
his/her  position  within  the  agency.  The 
responses  and  findings  of  the  focus  group  may 
result  in  limitations  which  impact  the  validity 
and  reliability  of  the  data. 

Accent  knowledge:  Make  known  (in  the 

funding  application  process)  the  unique 
aspects  (cultural,  regional,  etc.),  of  your 
population  in  need.  The  uniqueness  of  your 
population  in  need  and  the  need  for  unique 
approaches  in  prevention  strategies  may  not 
be  known  to  your  funding  agencies.  For 
example,  many  HIV/AIDS  organizations  that 
develop  programs  for  people  of  color 
recognize  and  embrace  the  fact  that  a lot  of 
the  members  of  these  cultural  groups  are  from 


oral  cultures.  This  factor  has  implications  for 
evaluation.  If  a person  tends  to  respond  to 
questions  better  when  they  are  presented 
verbally,  then  he  or  she  may  not  be  likely  to 
respond  to  a four  - six  page  written  survey. 
Perhaps  this  individual  feels  that  he  or  she 
can  respond  more  accurately  and/or  elaborate 
on  response  through  a verbal  exchange. 
Having  knowledge  of  the  unique  cultural 
considerations  of  your  population  in  need, 
and  then  conveying  this  information  to 
Project  Officers  and  the  research  community 
at  large  is  a responsibility  of  organizations  in 
the  “Phase  Three”  stage  of  evaluation. 

Accurately  reflect:  Understand  that  the 

further  the  distance  from  the  population 
in  need  (either  physically  or  philosophically) 
the  greater  the  potential  for  evaluation 
concern.  Organizations  in  “Phase  Three” 
must  always  continue  to  make  certain  that 
they  are  accurately  reflecting  the  experiences 
and  concerns  of  their  population  in  need. 
Additionally,  organizations  in  “Phase  Three” 
must  continue  to  appreciate  the  evaluation 
limitations  of  their  constituents  and  strive  to 
assist  them  whenever  possible.  For  example, 
the  “Phase  Three”  organization  may  represent 
over  100  organizations  in  Phase  One. 

There  may  be  organizations  which  are  using 
volunteers  to  disseminate  condoms  and 
brochures  on  the  streets  in  select 
neighborhoods  from  8pm  - 2am  daily.  The 
time  in  which  these  CBO’s  can  respond  to 
request  from  the  national  organizations,  and 
the  restrictions  of  data  collection  at  1pm  must 
be  understood  and  appreciated  by  the 
national  organization.  The  organization  in 
“Phase  'Three”  of  evaluation  then  has  the 
additional  burden  of  impressing  upon  CBOs 
the  need  to  complete  data  collection 
instruments  and  forward  them  to  their  offices 
accurately  and  in  a timely  fashion. 


glossary  of  key  terms 


Accountability:  Responsibility  of  program  staff  to 
provide  evidence  - to  sponsors,  boards  and  the 
community  among  others  — of  conformity  to 
program  specifications  and  fiscal  requirements. 

Activities:  The  day-to-day  tasks  involved  in 
program  planning  and  development  that  the 
member  of  the  population  in  need  participates  in 
and/or  are  impacted  by. 

Community  Analysis:  one  form  of  needs 
assessment  that  examines  the  characteristics  and 
health  problems  of  a community. 

Control  Group:  A segment  of  the  target  group 
not  receiving  intervention. 

Culture:  The  shared  values,  norms,  traditions, 
customs,  arts,  history,  folklore,  and  institutions  of 
a group  of  people. 

Cultural  Competence:  A set  of  academic  and 
interpersonal  skills  that  allow  individuals  to 
increase  their  understanding  and  appreciation  of 
cultural  differences  and  similarities  within, 
among,  and  between  groups.  This  requires  a 
willingness  and  ability  to  draw  on  community- 
based  values,  traditions,  and  customs  and  to  work 
with  knowledgeable  persons  of  and  from  the 
community  in  developing  focused  interventions, 
communications,  and  other  supports. 

Cultural  Diversity:  Differences  in  race,  ethnicity, 
language,  nationality,  or  religion  among  various 
groups  within  a community,  organization,  or 
nation.  A city  is  said  to  be  culturally  diverse  if  its 
residents  include  members  from  different  groups. 

Culturally  Appropriate:  Demonstrating  both 
sensitivity  to  cultural  differences  and  similarities 
and  effectiveness  in  using  cultural  symbols  to 
communicate  a message. 

Ethnic:  Belonging  to  a common  group  - often 
linked  by  race,  nationality,  and  language  - with  a 
common  cultural  heritage  and/or  derivation. 

Goals:  A statement  that  indicates  what  a program 
is  supposed  to  produce.  A goal  statement 
describes  the  intended  consequences  of  the 
program  being  developed. 

GO  AM:  An  acronym  for  Goals,  Objectives, 
Activities,  and  Milestones;  a planning  and 
evaluation  strategy  for  assessing  tasks  necessary  to 
complete  a project. 

Impact  evaluation:  Assess  the  immediate  effects 
of  a program. 

Experimental  Group:  Segment  of  the  target 
population  to  whom  an  intervention  is  delivered 


and  whose  impact/outcome  measures  are 
compared  with  the  control  group. 

Intervention:  An  planned  activity  designed  to 
produce  intended  changes  in  a target  population. 

Mainstream:  A term  that  is  often  used  to  describe 
the  “general  market,”  usually  refers  to  a broad 
population  that  is  primarily  White  and  middle 
class. 

Milestones:  A plan  to  evaluate.  Built  in  steps  in 
your  prevention  program  which  encourage  you  to 
evaluate  the  success  of  your  activities  toward  the 
overall  objective. 

Multicultural:  Designed  for  or  pertaining  to  two 
or  more  distinctive  cultures. 

Nationality:  The  country  where  a person  lives 
and/or  one  that  he  or  she  defines  as  a homeland. 

Objectives:  Tasks  that  must  be  completed  to 
achieve  the  goal. 

Outcome  Evaluations:  Designed  to  examine  the 
long-term  effects  of  the  programs  in  terms  of 
morbidity  and  mortality  rates 

Race:  A socially  defined  population  that  is 
derived  from  distinguishable  physical 
characteristics  that  are  genetically  transmitted. 

Reliability:  Refers  to  the  need  to  establish  the 
internal  consistency  of  the  instrument;  the  ability 
to  yield  similar  results/responses  from  the  same 
respondent  (in  an  unchanging  situation)  each 
time  the  instrument  is  used.  A good  example  is  a 
knowledge,  attitude  and  beliefs  survey  instrument 
which  is  tested  and  time  again  on  the  same  kind 
of  respondent  (freshman  students  at  a local 
community  college)  and  gamers  the  same  results. 

Reliable  Measures:  A measure  on  which  scores 
are  reproducible  in  repeated  administrations, 
assuming  relevant  factors  are  the  same. 

Validity:  Refers  to  the  degree  to  which  the 
instrument  actually  measures  what  it  purports  to 
measure.  Instruments  must  be  BOTH  reliable  and 
valid.  Reliability  does  not  guarantee  validity  (i.e., 
and  instrument  can  be  reliable  but  invalid); 
however,  an  instrument  which  is  not  reliable  can 
NOT  be  valid. 


appendix 


attachment  # 1 — 5W's  worksheet 


Use  the  5 W’s  to  describe  your  program.  When  developing  an  evaluation  plan  you  may  want  to 
use  the  following  form  as  a guide. 


w 


What 


When 


Where 


Task  Your  Program 


1.  To  whom  are  you  providing  prevention  services? 

2.  Describe  the  population  in  terms  of 
demographic,  need,  and  response  levels  to  pre- 
existing prevention  efforts. 

1.  What  specifically  is  your  prevention  program 
doing? 

a.  Are  you  trying  to  increase  knowledge 
regarding  behaviors  that  put  individuals  at 
risk  for  HIV/AIDS? 

b.  Are  you  trying  to  provide  condoms  and 
hopefully  teaching  the  proper  use  of 
condoms  to  a select  population. 

2.  Outline  specifically  what  it  is  you  are  doing  and 
the  rationale  for  each  using  this  approach  over 
another  — i.e.  are  you  distributing  condoms  in 

(reference)  because 

Describe  when  you  conduct  activities  related  to 
your  objectives  and  under  what  conditions. 

Describe  the  places  in  which  your  HIV/AIDS 
prevention  activities  occur,  i.e.  Do  you  conduct 
HIV/ AIDS  101  workshops  in  beauty  and  barber 
shops  between  the  time  frames  of  5pm-  9pm 
on  Friday  evenings. 

This  question  sets  the  evaluation  in  motion. 

What  are  the  particular  questions  the  funders  want 
you  to  answer  about  the  population  in  need. 
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Why 


attachment  # 2 — GOAM  worksheet 


Evaluation  Action  Steps  (Process) 

The  following  is  an  example  of  how  this  project  looks  using  the  GOAM  process  for  evaluation. 

GOAM  Group/Action  Steps 


Goals 

A statement  that  includes  what  a program  is 
supposed  to  produce  e.g.  outcomes.  A goal 
statement  describes  the  intended  consequences  of 
the  program  being  developed. 

0 

Tasks  that  must  be  completed  to  achieve  the  goal. 

Activity 

The  specific  tasks/function  undertaken  to 
accomplish  the  objective. 

Milestones 

A significant  point  in  the  development  towards 
the  goal. 

sample  evaluation  tools 


the  agency 


Note  to  the  Reader:  Basic  data  gathering  instrument  ( Phase  I). 

DEMOGRAPHICS: 

Gender:  □ Male  □ Female  □ Transgender 


Age: 

□ Under  12 

□ 32-40 

□ 61  and  under 

□ 13-22 

□ 41-50 

□ 23-31 

□ 51-60 

Ethnicity: 

□ African-American 

□ Asian/Pacific  Islander 

□ Caucasian 

□ Latina/o 

□ Native  American 

□ Other 

Have  you  ever  attended  an  AIDS  101  workshop  before?  □ yes  □ no 
What  did  you  learn  that  you  didn’t  know  before? 


the  agency  — condom  survey 


Note  to  the  Reader:  This  an  example  of  a culturally  specific  condom  survey.  You  may  use  it  to  collect  culturally 
specific  data  on  your  target  population /risk  group. 


l.Your  Ethnicity 
A.  Asian 


□ Cambodian 

□ Malaysian 

□ Chinese 

□ Pakistani 

□ East  Indian 

□ Thailander 

□ Filipino 

□ Vietnamese 

□ Indonesian 

□ Laotian 

□ Japanese 

□ Others 

B.  Asian  Pacific  Islander 

□ Guamaniam 

□ Samoan 

□ Hawaiian 

□ Others 

— 

2.  Your  Gender 

□ Male 

□ Female 

□ Transgender 

3.  Your  Marital  Status 

□ Single 

□ Married 

□ Divorced 

□ Other 

□ Partnered 

4.  Your  Sexual  Orientation 

□ Gay/Lesbian 

□ Straight 

□ Bisexual  □ MSM 

5.  Are  you  sexually  active? 

□ yes 

□ no 

6.  Do  you  know  what  condoms  are  used  for? 

□ yes 

□ no 

If  no,  why? 

□ Lack  of  information  □ Cannot  afford  to  buy 

□ Other  reasons 

7.  Do  you  know  how  to  use  condoms  properly? 

□ yes  □ no 


condom  survey 


cont'd 


If  no,  why?  □ Lack  of  information  □ Language  Barrier 

□ Other  reasons 

8.  Do  you  use  latex  condoms? 

□ yes  □ no 

If  no,  what  other  condom  do  you  use? 

□ Polyurethane  □ Lambskin 

9.  Do  you  use  another  kind  of  condom? 

□ yes  □ no 

10.  Do  you  use  water  base  lubricants  with  the  condoms? 

□ yes  □ no 

If  no,  what  kind  of  lubricants  do  you  use? 

□ K.Y  □ Oil  □ Vaseline  □ Spit 

11.  Do  you  know  where  to  get  condoms  and  lubricants  in  your  area? 

□ yes  □ no 

12.  Do  you  want  to  get  condoms  and  lubricants? 

□ yes  □ no 

I yes,  give  us  your  Zip  Code. 

13.  Area/Jocation  where  this  survey  was  taken. 

□ Chinatown  □ Uptown  □ Rizal  Center 

□ Edgewater/Rogers  Park  □ Other  

If  you  want  to  receive  condoms,  lubricants  and  other  information  about  safer  sex,  please 
contact  The  Agency  at  the  number  below: 


Phone:  1-888-000-0000 


Address:  1111  Street,  Suite  222 
City,  State  22110 


the  agenof 


Note  to  Reader:  Questions  for  Focus  Group /In-dept  Interviews  — - This  assessment  can  he  reworded  to  ask  clients 
about  condoms,  support  groups,  workshops,  etc. 


Please  fill  out  the  questionnaire  below  to  share  your  ideas  on  how  to  make  the  Exchange  and  the 
Harm  Reduction  Center  more  effective.  Thank  You 

What  types  of  syringes  (size,  etc.)  and  other  supplies  would  you  like  provided  at  the 
Needle  Exchange  site? 


What  do  you  think  are  the  best  times,  day  or  night,  to  conduct  Needle  Exchanges? 


What  are  the  most  convenient  and  safest  sites  for  Needle  Exchange? 


Would  you  use  a site  if  it  were  indoors? 


How  do  you  think  we  can  encourage  more  people  to  use  the  ex-changes? 


What  other  services  would  you  like  provided  at  the  site? 


□ Health  Services 

□ HIV  & Drug  Counseling 

□ Vocational  training 


□ Benefits  counseling 

□ Voter  registration 

□ Referrals  to  other  agencies 


the  agency  outreach  log 


Note  to  the  Reader:  The  tools  can  he  used  to  validate  collaboration  and  identification  of  the  usage,  distribution  and 
tracking  demographics. 


Name: 


Date: 


Outreach  Location: 


# OF  CLIENT  CONTACTS: 

1.  #IDU/DU (donotincludeMSM/IDU  here)  2.  # Sexual  Partner  (not  IDU) 

3.  # Total  MSM (includes  MSM  and  MSM/IDU,  not  IDU)  4.  Population  Not  Known  _ 

AGE: 

Under  25  Over  25  Unknown 


GENDER: 

Female Male TG 

ETHNICITY: 

African  American Latino/Latina White 

Asian/Pacific Native  Amer. Other/Unk 

SEXUAL  ORIENTATION: 

Gay/Bisexual  Males Lesbian/Bisexual  Females 

Fleterosexual  Males Heterosexual  Females 


SUPPLIES  DISTRIBUTED: 

# Condoms # Bleach  Bottles. 

# Lubricants # Dental  Dams  _ 

TOTAL  # OF  REFERRALS 

Services  Referred  To: 

HEALTH  SERVICES 

# HIV  Testing  

# TB  Screening  

# STD  Screening 

# Detox  or  Treatment 

Medical  Care  Clinic 


#Alcohol  Wipes 
# Offer 


SOCIAL  SERVICES 

Needle  Exchange 

Foo&Food  Banks 

Shelter/housing 

GA/S  S I/A  FDC 

HIVIAIDS  Support 


Other  Medical  or  Social  Services  ( Specify ) 


the  agency  action  plan 


Daily  ACTIVITIES  LOG  Date: Session: Attendance: 

N otes: 

Date: Session: Attendance: 

Notes: 


Date: Session: Attendance: 

Notes: 

Date: Session: Attendance: 

Notes: 


Date: Session: Attendance: 

Notes: 

Date: Session: Attendance: 

Notes: 


Date: Session: Attendance: 

N otes : 

Date: Session: Attendance: 

Notes: 


Date: Session: Attendance: 

N otes : 

Date: Session: Attendance: 

Notes: 


Date: Session: Attendance: 

N otes : 


Date: Session: Attendance: 

N otes: 


IP 


pre-post  test 


Name  (optional):  Date: 

1.  HIV  is  transmitted  from  human  to  human  by: 

a.  sexual  contact 

b.  blood-to-blood  contact 

c.  insect  bites 

d.  both  A and  B above 

2.  Which  of  the  following  body  fluids  does  not  transmit  HTV? 

a.  blood 

b.  semen 

c.  saliva 

d.  vaginal  secretions 

3.  The  single  best  means  of  prevention  from  becoming  infected  with  HIV  is: 

a.  to  only  have  sex  with  someone  you  know 

b.  wash  your  hands  before  and  after  any  sexual  contact 

c.  abstaining  from  any  sexual  contact  or  drug  use 

d.  to  use  a condom  always 

4.  AIDS  stands  for:  (Check  the  correct  answers.) 

a.  acquired  intravenous  deficiency  syndrome 

b.  acquired  immune  deficiency  syndrome 

HIV  stands  for: 

a.  Human  Immunodeficiency  Vaccine 

b.  Human  Immunodeficiency  Virus 

5.  What  is  the  main  role  of  the  white  blood  cells? 

a.  to  send  pain  messages  to  the  brain 

b.  to  decide  the  sex  of  an  unborn  child 

c.  to  make  more  growth  hormones 

d.  to  fight  off  infection  in  the  body 

6.  What  is  the  main  role  of  the  T-helper  cells? 

a.  to  make  more  protein  in  the  muscles 

b.  to  make  adrenaline 

c.  to  organize  the  body  as  immune  defenses 

d.  to  shut  off  the  body's  pain  signals 


pre-post  test  — cont'd 


7.  What  happens  to  the  white  blood  cells  after  HIV  infection? 

a.  they  turn  into  red  blood  cells 

b.  they  are  destroyed  by  the  virus 

c.  they  become  cancer  cells 

d.  they  grow  out  of  control 

8.  Why  is  HTV  different  from  other  viruses? 

a.  it  uses  the  immune  system's  cells  to  grow 

b.  both  animals  and  humans  can  get  it 

c.  it  dies  at  once  from  touching  white  blood  cells 

d.  it  can  be  killed  by  T-helper  cells  only 

9.  What  is  an  opportunistic  infection? 

a.  infection  that  waits  until  the  body  reaches  a certain  age 

b.  infection  that  can  be  cured  by  aspirin  and  rest  in  bed 

c.  infection  that  lives  on  red  blood  cells 

d.  infection  that  takes  advantage  of  damaged  immune  system 

10.  How  is  HTV  passed  on  to  another  person  during  sexual  intercourse? 

a.  through  sweat  from  body  contact 

b.  through  saliva  when  kissing 

c.  through  semen,  vaginal  fluids,  and  menstrual  blood 

d.  through  semen,  vaginal  fluids,  and  body  sweat 

11.  Which  sexual  practices  can  transmit  HIV? 

a.  anal  sex  only 

b.  vaginal,  anal,  and  oral  sex 

c.  anal  and  oral  sex  only 

d.  vaginal  sex  only 

12.  How  is  HTV  spread  through  needle  use? 

a.  illegal  IV  drugs  carry  the  virus 

b.  the  virus  can  cling  to  the  outside  of  a needle  before  it  is  used 

c.  infected  blood  in  used  needles  can  by  injected  into  the  next  user 

d.  soaking  used  needles  in  bleach  will  kill  everything  but  the  AIDS  virus 

13.  What  are  the  two  ways  HIV  test  results  are  handled  in  labs? 

a.  public,  open 

b.  anonymous,  confidential 


pre-post  test  — cont'd 


c.  secret,  goes  into  a state  data  bank 

d.  shown  to  the  doctor,  then  destroyed 

14.  What  is  the  most  widely  known  drug  in  treating  AIDS? 

a.  amoxicillin 

b.  AZT 

c.  ARC 

d.  ventolin 

15.  List  three  things  persons  with  AIDS  can  do  to  take  care  of  their  overall  health? 

a. 

b. 

c. 

16.  What  is  the  surest  way  to  keep  from  getting  infected  with  HIV? 

a.  use  a condom  during  sex 

b.  never  shoot  IV  drugs 

c.  abstain  from  all  high  risk  behaviors 

d.  avoid  the  use  of  public  toilets 

17.  Why  should  condoms  be  used  during  sexual  intercourse? 

a.  you  can  pass  on  a disease  by  having  unprotected  sex 

b.  condoms  are  an  important  disease  prevention  method 

c.  everyone  must  take  responsibility  for  stopping  AIDS 

d.  all  of  the  above 

18.  Which  type  of  condom  is  best  for  protection  against  HIV? 

a.  lambskin 

b.  tissue 

c.  latex 

d.  seal  skin 

19.  The  best  way  to  keep  from  getting  infected  with  HIV  from  needles  is  to  stop  using  them 
completely.  But  if  a person  keeps  on  using  needles,  what  is  the  next  best  thing  to  do? 

a.  don't  share  needles  with  strangers 

b.  wash  hands  after  every  use 

c.  share  needles  only  with  people  you  know 

d.  always  clean  needles  in  bleach  before  using 


pre-post  test  — cont'd 


20.  What  does  the  term  "universal  precautions"  mean? 

a.  standard  disease  protection  practices 

b.  a brand  of  condoms 

c.  standard  ways  to  decide  who  has  AIDS 

d.  a type  of  plastic  glove 

21.  The  First  HIV  antibody  test  became  available  in  what  year? 

a.  1980 

b.  1985 

c.  1987 

d.  1991 

22.  In  general  when  a person  wants  to  be  tested  for  HIV  Antibodies  what  must  be  obtained 
from  the  by  the  agency  doing  the  testing? 

a.  ten  dollars 

b.  informed  Consent 

c.  demographic  information 

d.  proof  of  age 

23.  If  someone  was  exposed  to  the  HIV/AIDS  virus,  what  is  the  maximum  amount  of  time  it 
could  take  for  the  virus  to  show  up  in  their  blood,  if  were  tested. 

a.  6 months 

b.  2 years 

c.  5 years 

d.  10  years 

24.  An  infected  woman  can  transmit  HIV  to  unborn  to  newborn  baby: 

a.  through  breast  feeding 

b.  during  the  birth  process 

c.  during  pregnancy 

d.  all  of  the  above 

25.  The  HIV  virus  can  be  transmitted  from  needle  use  by: 

a.  tattooing  and  body  piercing 

b.  donating  blood 

c.  steroid  use 

d.  Both  A and  C 


session  evaluation  form 


Name  (optional) i Date: 

Outreach  location: 


Your  comments  are  important  and  useful  to  our  program.  Please  take  a fezu  minutes  to  fill  out  this  evaluation.  We  will 
take  your  comments  into  consideration  for  future  sessions.  Thank  You. 


1.  How  much  did  you  know  about  HIV  or  STD's  before  this  session? 

□ A lot  Z Some  Z Very  Little  Z Nothing 

2.  How  much  did  this  session  help  you  to  understand  HTV? 

□ A lot  DSome  □ Very  Little  □ Nothing 

3.  Are  you  now  able  to  identify  the  ways  HIV  and  STD’  are  passed  on  to  another  person? 

□ Yes  □ No  □ Unsure 

4.  Are  you  now  able  to  identify  "high-risk"  behaviors  for  HTV  and  STD's? 

□ Yes  □ No  □ Unsure 

5.  Do  you  feel  that  you  will  change  some  of  your  "high-risk"  behaviors  as  a result  of  the 
information  you  have  received? 

□ Yes  □ No  □ Unsure 

6.  Do  you  feel  that  the  instructors  had  good  knowledge  of  the  subject? 

□ Yes  □ No  □ Unsure 

7.  Do  you  feel  All.  topics  were  thoroughly  covered? 

_J  Yes  □ No  □ Unsure 

8.  Was  the  material  presented  valuable  to  you? 

_ Yes  □ No  □ Unsure 

9.  Did  you  feel  comfortable  asking  questions  or  making  comments? 

□ Yes  □ No  □ Unsure 


continued  on  next  page 
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session  evaluation  form 


10.  Have  you  been  tested  for  HIV? 

□ Yes  □ No 

If  no,  would  be  more  willing  to  be  tested,  now 
that  you  have  attended  HIV  classes? 

□ Yes  □ No  □ Unsure  Why?. 


session  ( continue  for  Juveniles  only) 

Are  there  any  other  HIV  topics  that  you  would  like  to  learn  about  in  a future  program ? If  you  were  to  test 
positive  for  HIV  tomorrow,  what  would  you  do?  How  do  you  think  your  jriends  and  family  would  react  if  you 
told  them  you  were  HIV  positive?  How  would  you  react  to  learning  that  a friend  or  family  member  was  HIV 
positive? 

l.When  you  engage  in  sexual  activity,  how  often  do  you  use  a condom? 

□ Always  □ Most  times  □ Sometimes  □ Never 

Why  or  Why  Not? 


3.  After  your  release,  would  you  be  interested  in  helping  us  make  a video  about  HIV 
and  AIDS? 

Yes  or  No,  Why? 

4. What  part  of  the  HTV/AIDS  classes  did  you  enjoy  the  most? 


revised  10/97  EVAL.  CDC/Juv  only 


presentation  evaluation 


Please  fill  out  the  following  evaluation  and  pass  in  at  the  end  of  the  presentation. 


Title 

Date:  

Site: 

Presenter  (s): 


PROFILE  OF  AUDIENCE; 


Please  put  a check  next  to  the  category  which  describes  yourself: 


□ M DF 

□ African-American  □ Latino 

□ Euro-American  □ Cape  Verdean 

□ Haitian  □ Asian 


□ Native  American 

□ Brazilian 

□ Other  (Specify 


1.  How  would  you  rate  the  overall  presentation? 

□ poor  □ average  □ good 


□ very  good  □ excellent 


2.  Was  the  material  presented... 

□ inappropriate  □ appropriate 


3.  How  would  you  rate  your  opportunity  to  participate? 

□ poor  □ average  □ good  □ very  good 


□ excellent 


4.  What  was  most  helpful  about  the  presentation? 


5.  Do  you  have  any  suggestions  or  recommendations  about  the  presentation? 


6.  Are  there  specific  topics  you  would  like  to  see  addressed  in  future  seminars? 
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individual  staff  evaluations 


Please  mark  the  appropriate  rating  for  the  following  presenter(s),  using  “P"  for  poor,  " AVER " for  average,  "G"  for  good, 
" VG"  for  very  good,  and  " EX " /or  excellent: 


John  Doe  1 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

Jane  Doe  1 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

Jane  Doe  2 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

John  Doe  2 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

John  Doe  2 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

Jane  Doe  3 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

John  Doe  3 

□ P 

□ AVER 

□ G 

□ VG 

□ EX 

monthly/quarterly  report 


Race/Ethnicity: 

□ Nat.  Am./Ala.  Nat. 

□ Haitian  (any  race) 

□ Other/Unknown 

MEN 

0-14  

15-19  

20-29  

30-39  

40-44  

45+  

Undetermined  

— — — 

MEN  SUBTOTAL  

— — _ __ 

WOMAN 

0-14  

15-19  

20-29  _____ 

30-39  

40-44  

45+  

Undetermined  

— — — 

WOMAN  SUBTOTAL  

— __  — 

TOTAL  CONTRACTS  

_ Asian/  Pacific  Is.  __  Black  (non-H/H) 

_ Hispanic  (any)  □ White 


(total  men) 


(total  men) 
(grand  total) 


BEHAVIOR  RISK  EXPOSURES 

High  Risk  Behavior 

IDUs:  I dtVj 

MSM:  

MSM  and  IDUs:  

Hetero.  Sex  Partners  of  MSM:  

Sex  Partners  of  IDUs:  

Sex  Partners  of  HIV+s,  PWAs:  

Total  High  Risk  a t > 

Increased  Risk  Behavior 

Non-monogamous  Heterosexuals:  

Substance  Abusers:  

Total  Increased  Risk  

Undetermined/No  Identified  Risk: 

Total  Contacts  in  Risk  Groups  

Total  # of  condoms  distributed:  

Total  # of  brochures  distributed:  

Total  # of  Referrals  to:  

STD  Clinics:  

HIV  C/T:  

TB  Clinics:  

Prevention  Case  Management:  

Total  Referrals:  


% 


Signature  of  Project  Officer 


TARGET  GROUPS  # % 

Persons  who  barter  sex:  

Persons  infected  with  STDs:  

At-Risk  Youth:  

Persons  Infected  with  TB:  

Homeless  Persons:  

Migrant/Seasonal  Farmworkers:  

Inmates/Paroles:  

General  Population/Others:  

EDUCATION  & OUTREACH  CONTACTS 

Total  number  education  sessions:  

Total  contacts  from  education:  

Pre-  and  Post-tests  conducted  in:  ed  sessions 

Participant  evals.  conducted  in:  ed  sessions 

Total  number  of  outreach  hours:  

Total  contacts  from  outreach:  

Total  number  1-on-l  contacts:  

Materials/Referrals/ Assessments:  

MATE  RIALS/REFERRALS/ ASSIGNMENTS 

Behavior  Assessment 

1.  # reporting  intended  change:  ___  

2.  # reporting  actual  change:  

Total  # of  Pre/Post-tests  conducted: 

1.  Average  Pre-Test  score:  

2.  Average  Post-Test  score:  

3.  Average  Change  (+/-):  

Date 


Please  copy  and  MAIL  IN. 


evaluation  document  survey 


Please  complete  and  return  this  survey  to  NMAC’s  Technical  Assistance  Division.  This  will  assist  us  in  responding  to  your 
Technical  Assistance  needs  in  the  area  of  Program  Evaluation.  MAIL  THIS  SURVEY  TO:  Technical  Assistance  Division 
National  Minority  Aids  Council,  1931 13th  Street,  NW,  Washington,  DC  20009-4432. 

Agency  Name:  ( optional ) 

Address  (city /state/ zip  code) 

Telephone  # (optional) 


Do  you  consider  your  organization  (mark  those  that  apply) 

□ National  □ Regional  □ Community  Based  □ Church  Affiliated 

□ Government  □ Not  for  profit  □ Private  □ other 

Does  your  agency  have  HIV  Prevention  programs?  □ Yes  □ No 
Number  of  staff  working  in  HIV  Prevention  . 

Which  populations  do  you  serve?  (please  mark  all  that  apply) 

□ African  American  □ Latino  □ Asian  & Pacific  Islander 

□ Native  American  □ Caucasian  □ MSM 

□ Women  □ IDU  □ Youth 

□ Incarcerated  □ other 


Did  you  read  the  entire  document?  □ Yes  □ No 

Was  the  material  useful?  □ not  useful  □ somewhat  useful  □ useful  □ very  useful 

According  to  this  material,  please  rate  the  phase  you  think  your  agency  is  in 

□ Phase  1 □ Phase  2 □ Phase  3 


Are  you  currently  evaluating  your  programs?  □ Yes  □ No 
Do  you  have  an  outside  evaluator?  □ Yes  □ No 

What  type  of  evaluation  do  you  use?  □ Process  □ Outcome  □ Impact 


□ None 


If  you  were  to  receive  technical  assistance  in  Evaluation  please  check  off  what  you  are 
currently  doing  (Current)  and  rank  your  priorities  in  the  right  column  (1  being  the  most 
important). 


Current 

Evaluation 

FUTURE 

Theories  of  Evaluation 

Developing  evaluation  Instruments 

Methods  in  Evaluation  Design 

Innovative  Data  Collection  Strategies 

Culturally  Sensitive  Evaluation  Techniques 

Writing  Evaluation  Plans 

other: 

J 


( \ 

■■ 

■■ 

NATIONAL 

MINORITY 

AIDS 

COUNCIL 


1931  13th  Street,  NW  Washington,  DC  20009  P (202)483.6622  F (202)483.1135 


